Immersive virtual reality (VR), augmented reality (AR), or mixed reality (MR) systems typically utilize a head mounted display (HMD) that presents stereoscopic imagery to the user so as to give a sense of presence in a three-dimensional (3D) scene. A typical HMD is designed to produce a stereoscopic image over a field-of-view that approaches or is equal to the field-of-view of a human eye, which is approximately 180°. For example, the field-of-view of commercial HMDs is currently 100-110°. Multiple users can interact with each other in the same 3-D scene produced by an immersive VR, AR, or MR system. For example, users can interact with each other using 3D video conferencing, while co-watching movies or YouTube videos in a virtual theater, taking a virtual hike through a region in Google Earth, or while sitting in a virtual 3D classroom listening to a lecture by a (real or virtual) professor. Some immersive VR, AR, or MR systems use a camera to capture images of the users, which are then inserted into the virtual 3-D scene. In some cases, such as systems that implement avatar-based representations, the “camera” is an abstraction that is used to indicate a point of view from which the scene is rendered. In cases where the scene includes a user wearing an HMD, the camera is typically located outside of the HMD and renders or captures the scene from an external point of view, relative to the user wearing the HMD. Consequently, the user's faces, and in particular the user's eyes, are obscured by the HMD so that the images of the users that are inserted into the virtual 3-D scene have a disconcerting “brick-in-the-face” appearance. Consequently, the HMDs prevent the users from making eye contact during virtual interactions, which can disrupt the sense of immersion and social connection between the users in the virtual 3-D scene.